1. Field of the Invention
This invention relates to the measurement of the shape of an object or scene, and more particularly, to a method and apparatus to generate three-dimensional (3-D) coordinates of points of a surface, as well as a 3-D model of an object or scene, by a freely moving sensor whose position is not measured by any position tracking device, and which performs the measurement without contacting the object. The object and sensor are permitted free movement relative to one another, and therefore either the object or the sensor or both may be moving freely simultaneously.
2. Description of the Prior Art
Range sensors or scanners are commonly used for non-contacting measurement of the surface geometry of an object or scene (multiple objects). Typically the range-sensor consists of a range-sensor head and a mechanical positioning device, as shown schematically in FIG. 1 (prior art), for the particular case where the range-sensor head consists of a laser source connected to a camera. Usually, the range-sensor head acquires profiles of 3-D coordinates of the surface being measured. In order to acquire sufficient data to describe an object surface, the range-sensor head must be translated or rotated relative to the object surface to acquire numerous profiles, which will later be integrated into a single object model. The object may also be translated or rotated relative to a stationary or moving range-sensor head. In all cases, the movement of the moving body or bodies, the range-sensor head, the object, or both, is carried out by mechanical positioning devices which move the range-sensor head or object, either by predefined increments or to unknown positions which are measured by a position sensor (not shown) on the positioning device, during the data acquisition. Accurate information of the incremental movement of the moving bodies is required in order to integrate all profiles collected from the same position of the range-sensor apparatus.
However, for large objects, such as car-body panels, engines, machines, or statues, whose dimensions exceed the working dimensions of the rotation or translation stages, or for immovable objects, or objects which are difficult or not desirable to manipulate, such as human bodies and archaeological artifacts, the method of moving the object relative to a stationary range-sensor head cannot be used. Furthermore, the scanning of such objects by moving the range-sensor head, cannot be carried out with the mechanical positioning device at a single position. The mechanical positioning devices are limited in their range of movement, and cannot move the range-sensor head completely around the object.
One approach to measure such objects is to first take one series of scans across the object (usually moving along only one translation axis), as shown in FIG. 2a (prior art) and then reposition the entire scanning apparatus (the mechanical positioning device which holds the range-sensor head) several times to new positions or viewpoints to acquire a series of profiles or a range-image at each viewpoint, as shown in FIG. 2b (prior art). The different range images or views, which consist of many profiles each and have large movement between viewpoints, can then be integrated into a single reference frame if there is sufficient overlap between range images, as shown in FIG. 3a-3c (prior art). However, one problem with this method is that estimates of the movement between viewpoints must be obtained before or during the data acquisition process, as these estimates are necessary to carry out the integration of the range-views. Another problem with this method is that very complex planning is needed to measure the full object surface and ensure that some overlap between views is obtained. Furthermore, for many environments, it is not possible to move translation or rotation devices to the site of the object, or to several different viewpoints around the object. The use of mechanical positioning devices is also not possible in measuring confined interior surfaces of complex shaped objects.
There have been several approaches to measure the surface geometry of objects without mechanical translation and rotary stages. These employ a hand-held range-sensor head, which would be continuously swept over the object surface to perform a scanning operation.
One device sold by 3D Scanners Ltd., of London, England, employs a range-sensor head attached to mechanical linkages or arms which are instrumented with rotary position sensors. The range-sensor head is hand-held and its motion is controlled by the user. The device allows a more continuous scanning and a more free access to the surface of an object than with translation and rotation stages by permitting simultaneous translations and rotations of the range-sensor head in all six degrees of freedom (three translations, three rotations). However, the device is highly limited by the lengths and possible orientations of the mechanical arms, and can therefore only accommodate small objects, or the apparatus requires repositioning and complex planning for large objects. The mechanical linkages and position sensors also add to the cost.
Another apparatus employs a non-mechanical magnetic-field tracking device to determine the position of the sensor-head, (see Fisher, R. B., Fitzgibbon, A., Gionis, A., Wright, M. and Eggert, D. xe2x80x9cA Hand-Held Optical Surface Scanner for Environmental Modeling and Virtual Realityxe2x80x9d, Virtual Reality World ""96 Conference, pp. 1-16, Stuttgart, Germany, 1996; and New Zealand Patent 293713 (Polhemus)), and if necessary, the target object (see U.S. Pat. No. 5,886,775, xe2x80x9cNoncontact digitizing imaging systemxe2x80x9d) during scanning. The device has eliminated the more bulky mechanical positioning and position sensing devices, and permits continuous sensor-head movement by hand to facilitate the range data acquisition process. However, because of the use of magnetic-field sensors, the system is restricted to non-ferrous environments. It is also costly to include the position tracking devices. Furthermore, in the case where the object is moving, contact must be made with the object to fix a position sensor to it. The magnetic-field position tracking system also has a limited range.
A third apparatus uses a non-mechanical optical position sensor to track the position of the range-sensing head during scanning (see U.S. Pat. No. Re. 035,816, xe2x80x9cMethod and apparatus for three-dimensional non-contact shape sensingxe2x80x9d, and U.S. Pat. No. 5,198,877 of the same title). Three point light sources (referred to as pilot lights) located on the range-sensing head emit light one at a time, using strobing. The emitted light is received by three multiplexed light sensors (operated one at a time) to determine the position of the range-sensor head. A major drawback of the apparatus is that it requires unobstructed lines of sight between the pilot lights located on the range-sensor head, and the pilot-light sensors. This would be restrictive or prohibitive, in manipulation of the range-sensor head when scanning completely around objects, or inside confined cavities. Another drawback of this apparatus is the use of components used to generate and detect light points on a surface for the purposes of sensor-head position tracking only. These components are separate from the range-sensing light source and receiver and add considerable cost and complexity to the system.
A fourth device and method is the only one known to the inventors to involve a fully unconstrained sensor without positioning devices [see Hebert P., and Rioux, M. xe2x80x9cToward a hand-held laser range scanner: integrating observation-based motion compensationxe2x80x9d, Proc Three-Dimensional Image Capture and Applications: SPIE-3313, pp. 2-13, 1998; and international patent application no. PCT/CA98/00324, publication no. WO 98/45817). The method requires two separate scans of the same region by the range-sensor head, to acquire two sets of profiles having an angle with respect to one another, and thus to obtain a collection of crossing profiles. Each single profile is considered to be acquired from a different unknown viewpoint. An algorithm adjusts each profile position and orientation separately to minimize the spacing between intersecting profiles. While the method eliminates the use of position sensors, the requirement of two separate scans at an angle to one another requires complex planning of the scanning path, and an awkward scanning process by a user when the device is hand-held. Alternatively, the two sets of profiles are acquired simultaneously by the range-sensor head, using two rotating mirrors which reflect two projected planes of light from two light sources. This adds considerable cost and complexity to the system.
The invention has the following objects, although not all embodiments of the invention may achieve all objects of the invention.
An object of the present invention is to allow non-contact measurement of the surface information of an object or scene using a physically unconstrained range-sensor head, without any mechanical positioning devices or mechanical position-measurement devices. This would allow the device to be compact and would permit measurement of large, immovable objects of complex shape, interior surfaces and surfaces in confined spaces, which do not allow access of a range-sensor equipped with mechanical positioning devices.
Another object of the present invention is to provide a method of non-contact scanning an object to measure surface information without the need to fix the range-sensor head position at accurately known incremental locations, to measure the range-sensor head position at incremental locations, or to fix the entire scanning apparatus at planned positions. This would facilitate the acquisition of range data by reducing the planning of the scanning.
Another object of the present invention is to provide an apparatus and method to measure surface information of an object or scene without contacting the object, by a range-sensor without having to know the position of the range-sensor head or measure its position in 3-D space during the acquisition of surface information. This suggests another object of the invention, namely to provide an apparatus and method to measure surface information of an object or scene without any position tracking or measuring devices, whether physical, magnetic, optical based or otherwise, to track the position of the range-sensor head, which may be freely moving in space.
Another object of the present invention is to provide an apparatus and method to measure surface information using continuous motion rather than in stages, to eliminate complex planning of the operation by the user.
Scanning with the present invention could be accomplished by a hand-held range-sensor head, where the user would continuously sweep the range-sensor head over the object surface. Alternatively, a physically unconstrained range-sensor head could be airborne. An unconstrained range-sensor which would eliminate the problem of requiring knowledge of the relative range-sensor head movement with respect to the object, would also permit measurement of continuously moving objects with unknown positions.
A further object of the present invention is to provide an apparatus and method to measure the surface information of an object by a range-sensor head, where the object or range-sensor head may be freely moving in 3-D space or both the object and sensor head may be freely moving simultaneously in 3-D space at any time during the surface measurement, where surface measurement is made without contacting the surface by the range-sensor, and where the positions in 3-D space of the range-sensor head and surface are not known and not measured during the acquisition of surface information, except for the range measurement itself of the relative distance of the surface to the range sensor head by the range-sensor head.
Another object of the present invention is to provide an apparatus and method to measure surface information of an object or scene without contacting the object, using a simple system, having no moving parts, few or no signals to measure other than those provided by the optical sensor which measures range (the distance from the object to the range sensor), which in one embodiment is a laser-camera unit. In certain embodiments of the present invention, this suggests that no analog/digital (A/D) converter would be necessary to measure such signals.
Another object of the present invention is to provide an apparatus and method to measure surface information of an object or scene using a simple system having low cost.
Another object of the present invention is to provide an apparatus and method to measure surface information of an object or scene, where the entire apparatus is highly portable.
Another object of the present invention is to provide an apparatus and method to measure surface information of an object or scene, where there are no restrictions on the materials in the environment surrounding the sensor, or on the ambient lighting.
An optional object of the invention is to acquire other information about the surface of an object, such as color intensity, gray-level intensity, temperature, etc. In the invention, any sensor-head which acquires surface information may therefore be used, i.e. not just a sensor which measures only range.
In a preferred embodiment of the invention, the range-sensor head has a projector which simultaneously projects a few multiple lines, and a charged-couple-device (CCD) camera, coupled to a computer using a framegrabber. The computer, framegrabber and computer programs containing the processing algorithms are the only other primary elements of the preferred apparatus. Some other less preferable embodiments may omit the framegrabber. A computer monitor is an optional component used to interface with the user, and to display the object geometry in real time and after processing.
A few profiles of three-dimensional coordinates of points of the object surface are acquired at a single range-sensor head viewpoint. In a preferred embodiment of the invention, this is achieved by capturing in a single camera image the light contours or profiles produced by projecting the lines of light simultaneously onto the object surface. A mathematical geometric surface can be fit to the few profiles of a view, and interpolation on the surface between the profiles can be performed. This is done to permit matching of adjacent surfaces fit to points of adjacent views.
The apparatus and method provide an integration of all range data acquired by the range sensor head at different unknown and unmeasured viewpoints, into a single set of points in a single reference frame and into a single geometric model, by the matching of adjacent surfaces fit to points of adjacent views, and by applying the transformations determined during the matching process. The method of the invention applies equally to matching any subset of the acquired views, where the subset is a sequence of overlapping views which are not necessarily adjacent in the originally acquired sequence of views.
The method of this invention applies equally to acquisition of points in a more random fashion than by profiles. In this case, a geometric surface can still be fit to the points although they do not belong to one of a few profiles.